Collaborative Robotic Systems Research Articles

Physical Ergonomics Monitoring in Human–Robot Collaboration: A Standard-Based Approach for Hand-Guiding Applications

Human–robot collaboration stands as one of the research frontiers in industrial applications due to the possibility for human operators to be supported by robots in carrying out their tasks in a shared workspace. However, advances in this field can be slowed down by the lack of standards regarding the safety and ergonomics of such applications. This article aims at reducing this gap by presenting an adaptation of the standard ISO 11228-3 for the ergonomic evaluation of hand-guiding applications through the OCRA index. This innovative methodology is innovatively applied to a drilling application in which a human operator hand-guides a collaborative robotic system consisting of a Franka Emika Panda robot, a force/torque sensor and an IMU suit to track the motion of the operator’s body. The SaRAH app, a MATLAB 2020a-based software tool developed on purpose, implements the ergonomic assessment procedure, allowing the proper redesign of the working shift (offline mode) or providing the worker suggestions to improve his/her behavior (online mode) so as to reduce the ergonomic risk.

Playing Checkers with an Intelligent and Collaborative Robotic System

Collaborative robotics represents a modern and efficient framework in which machines can safely interact with humans. Coupled with artificial intelligence (AI) systems, collaborative robots can solve problems that require a certain degree of intelligence not only in industry but also in the entertainment and educational fields. Board games like chess or checkers are a good example. When playing these games, a robotic system has to recognize the board and pieces and estimate their position in the robot reference frame, decide autonomously which is the best move to make (respecting the game rules), and physically execute it. In this paper, an intelligent and collaborative robotic system is presented to play Italian checkers. The system is able to acquire the game state using a camera, select the best move among all the possible ones through a decision-making algorithm, and physically manipulate the game pieces on the board, performing pick-and-place operations. Minimum-time trajectories are optimized online for each pick-and-place operation of the robot so as to make the game more fluent and interactive while meeting the kinematic constraints of the manipulator. The developed system is tested in a real-world setup using a Franka Emika arm with seven degrees of freedom. The experimental results demonstrate the feasibility and performance of the proposed approach.

A new automatic method for demoulding plastic parts using an intelligent robotic system

Nowadays, there are many different industrial processes in which people spend several hours performing tedious and repetitive tasks. Furthermore, most of these processes involve the manipulation of dangerous materials or machinery, such as the toy manufacturing, where people handle ovens with high temperatures and make weary physical effort for a long period of time during the process. In this work, it is presented an automatic and innovative collaborative robotic system that is able to deal with the demoulding task during the manufacturing process of toy dolls. The intelligent robotic system is composed by an UR10e robot with a RealSense RGB-D camera integrated which detects the pieces in the mould using a developed vision-based algorithm and extracts them by means of a custom gripper located and the end of the robot. We introduce a pipeline to perform the demoulding task of different plastic pieces relying in the use of this intelligent robotic system. Finally, to validate this approach, the automatic method has been successfully implemented in a real toy factory providing a novel approach in this traditional manufacturing process. The paper describes the robotic system performance using different forces and velocities, obtaining a success rate of more than 90% in the experimental results.

Mixed Reality for Safe and Reliable Human-Robot Collaboration in Timber Frame Construction

In the field of construction, human-robot collaboration and mixed reality (MR) open new possibilities. However, safety and reliability issues persist. The lack of flexibility and adaptability in current preprogrammed systems hampers real-time human-robot collaboration. A key gap in this area lies in the ability of the robot to interpret and accurately execute operations based on the real-time visual instructions and restrictions provided by the human collaborator and the working environment. This paper focuses on an MR-based human-robot collaboration method through visual feedback from a vision-based collaborative industrial robot system for use in wood stereotomy which we are developing. This method is applied to an alternating workflow in which a skilled carpenter lays out the joinery on the workpiece, and the robot cuts it. Cutting operations are instructed to the robot only through lines and conventional “carpenter’s marks”, which are drawn on the timbers by the carpenter. The robot system’s accuracy in locating and interpreting marks as cutting operations is evaluated by automatically constructing a 3D model of the cut shape from the vision system data. A digital twin of the robot allows the carpenter to previsualize all motions that are required by the robot for task validation and to know when to enter the collaborative workspace. Our experimental results offer some insights into human-robot communication requirements for collaborative robot system applications in timber frame construction.

Adaptive Learning based Upper-Limb Rehabilitation Training System with Collaborative Robot.

Rehabilitation training for patients with motor disabilities usually requires specialized devices in rehabilitation centers. Home-based multi-purpose training would significantly increase treatment accessibility and reduce medical costs. While it is unlikely to equip a set of rehabilitation robots at home, we investigate the feasibility to use the general-purpose collaborative robot for rehabilitation therapies. In this work, we developed a new system for multi-purpose upper-limb rehabilitation training using a generic robot arm with human motor feedback and preference. We integrated surface electromyography, force/torque sensors, RGB-D cameras, and robot controllers with the Robot Operating System to enable sensing, communication, and control of the system. Imitation learning methods were adopted to imitate expert-provided training trajectories which could adapt to subject capabilities to facilitate in-home training. Our rehabilitation system is able to perform gross motor function and fine motor skill training with a gripper-based end-effector. We simulated system control in Gazebo and training effects (muscle activation level) in Open-Sim and evaluated its real performance with human subjects. For all the subjects enrolled, our system achieved better training outcomes compared to specialist-assisted rehabilitation under the same conditions. Our work demonstrates the potential of utilizing collaborative robots for in-home motor rehabilitation training.Clinical relevance-The collaborative robot system is capable of providing safe and effective training comparable to specialized rehabilitation robots, enabling possibilities of convenient rehabilitation training at home.

Multi-objective task allocation for collaborative robot systems with an Industry 5.0 human-centered perspective

The migration from Industry 4.0 to Industry 5.0 is becoming more relevant nowadays, with a consequent increase in interest in the operators’ wellness in their working environment. In modern industry, there are different activities that require the flexibility of human operators in performing different tasks, while some others can be performed by collaborative robots (cobots), which promote a fair division of the tasks among the resources in industrial applications. Initially, these robots were used to increase productivity, in particular in assembly systems; currently, new goals have been introduced, such as reducing operator’s fatigue, so that he/she can be more effective in the tasks that require his/her flexibility. For this purpose, a model that aims to realize a multi-objective optimization for task allocation is here proposed. It includes makespan minimization, but also the operator’s energy expenditure and average mental workload reduction. The first objective is to reach the required high productivity standards, while the latter is to realize a human-centered workplace, as required by the Industry 5.0 paradigms. A method for average mental workload evaluation in the entire assembly process and a new constraint, related to resources’ idleness, are here suggested, together with the evaluation of the methodology in a real case study. The results show that it is possible to combine all these elements finding a procedure to define the optimal task allocation that improves the performance of the systems, both for efficiency and for workers’ well-being.

Visual-admittance-based model predictive control for nuclear collaborative robots

This paper presents a novel visual-admittance-based model predictive control scheme to cope with the problem of vision/force control and several constraints of a nuclear collaborative robotic visual servoing system. A visual-admittance model considering the desired image feature and force command in the image feature space is proposed. Moreover, a novel constraint scheme of the model predictive control (MPC) is proposed to cancel the overshoot in the interaction force control for most cases by taking the desired force command as the constraint of the proposed MPC. Via applying the robotic dynamic image-based visual servoing (IBVS) model, some other constraints, such as the actuator saturation, joint angle, and visual limits, can be satisfied simultaneously. The simulation results for the two-degrees-of-freedom (DOF) robot manipulator with an eye-to-hand camera are present to demonstrate the effectiveness of the proposed controller.

Development of a collaborative robotic system to polish leather surfaces

ABSTRACT Patina is the natural aging process that gives leather a unique, worn appearance. However, this patina effect can also be created artificially by using specific products to rub the leather surface. This research presents a new and innovative method to automate the process of creating patina on leather footwear using collaborative robots. These types of finishing tasks on medium size objects are delicate and often require adjustments by skilled craftsmen, adding complexity to the automation process. This new solution involves designing and implementing a collaborative tool specifically created to cultivate a patina finish on shoe surfaces. The system aims to speed up the process without compromising the quality of the finished product. In order to accomplish this task, a collaborative robot that can regulate contact force and a preexisting software feature for recording manual paths are utilized together with the skills of an expert craftsman. This enables the robot’s end effector to conform to the curved contours of the shoe. Furthermore, the tool’s path and position are modified based on the data from the CAD model to ensure precise outcomes. This study introduces a novel technique for automated and partially automated patina development in the shoe industry. This work presents a system integrated into a productive facility and it is currently being used.

Classification of primitive manufacturing tasks from filtered event data

Collaborative robots are increasingly present in industry to support human activities. However, to make the human–robot collaborative process more effective, there are several challenges to be addressed. Collaborative robotic systems need to be aware of the human activities to (1) anticipate collaborative/assistive actions, (2) learn by demonstration, and (3) activate safety procedures in shared workspace. This study proposes an action classification system to recognize primitive assembly tasks from human motion events data captured by a Dynamic and Active-pixel Vision Sensor (DAVIS). Several filters are compared and combined to remove event data noise. Task patterns are classified from a continuous stream of event data using advanced deep learning and recurrent networks to classify spatial and temporal features. Experiments were conducted on a novel dataset, the dataset of manufacturing tasks (DMT22), featuring 5 classes of representative manufacturing primitives (PickUp, Place, Screw, Hold, Idle) from 5 participants. Results show that the proposed filters remove about 65% of all events (noise) per recording, conducting to a classification accuracy up to 99,37% for subjects that trained the system and 97.08% for new subjects. Data from a left-handed subject were successfully classified using only right-handed training data. These results are object independent.

Подход к распределению работ в коллаборативной робототехнической системе с учетом модели рабочего пространства и динамического переназначения исполнителей

Подход к распределению работ в коллаборативной робототехнической системе с учетом модели рабочего пространства и динамического переназначения исполнителей

RESULTS OF THE RESEARCH ON THE ALLOCATION OF TASKS IN A COLLABORATIVE ROBOTIC SYSTEM WITH MINIMIZATION OF THEIR EXECUTION TIME

The article presents a generalized algorithm for allocation of tasks in a collaborative robotic system with minimizing the time of their execution (an algorithm of assigning the performers to work within the technological process), as well as the results of modeling the developed algorithm. The maximum value among the sums of the edge weights for the longest path from the source of the graph to its drain is used as the execution time of the technological process that needs to be minimized.

A Connective Framework for Social Collaborative Robotic System

Social intelligence in robotics appeared quite recently in the field of artificial intelligence (AI) and robotics. It is becoming increasingly evident that social and interaction skills are essentially required in any application where robots need to interact with humans. While the workspaces have transformed into fully shared spaces for performing collaborative tasks, human–robot collaboration (HRC) poses many challenges to the nature of interactions and social behavior among the collaborators. The complex dynamic environment coupled with uncertainty, anomaly, and threats raises questions about the safety and security of the cyber-physical production system (CPPS) in which HRC is involved. Interactions in the social sphere include both physical and psychological safety issues. In this work, we proposed a connective framework that can quickly respond to changing physical and psychological safety state of a CPPS. The first layer executes the production plan and monitors the changes through sensors. The second layer evaluates the situations in terms of their severity as anxiety by applying a quantification method that obtains support from a knowledge base. The third layer responds to the situations through the optimal allocation of resources. The fourth layer decides on the actions to mitigate the anxiety through the allocated resources suggested by the optimization layer. Experimental validation of the proposed method was performed on industrial case studies involving HRC. The results demonstrated that the proposed method improves the decision-making of a CPPS experiencing complex situations, ensures physical safety, and effectively enhances the productivity of the human–robot team by leveraging psychological comfort.

Human-Centered Design of a Collaborative Robotic System for the Shoe-Polishing Process

Demand for automated processes in the manufacturing industry is now shifting toward flexible, human-centered systems that combine productivity and high product quality, thus combining the advantages of automated and robotic systems with the high-value-added skills of operators and craftsmen. This trend is even more crucial for small and medium-sized enterprises operating in the “Made in Italy” fashion industry. The paper presents the study, simulation, and preliminary testing of a collaborative robotic system for shoe polishing that can reduce manual labor by limiting it to the finishing stage of the process, where the aesthetic result is fully achieved, with a benefit also in terms of ergonomics for the operator. The influence of process parameters and design solutions are discussed by presenting preliminary test results and providing hints for future developments.

Leader–Follower Dynamics in Complex Obstacle Avoidance Task

A question that many researchers in social robotics are addressing is how to create more human-like behaviour in robots to make the collaboration between a human and a robot more intuitive to the human partner. In order to develop a human-like collaborative robotic system, however, human collaboration must first be better understood. Human collaboration is something we are all familiar with, however not that much is known about it from a kinematic standpoint. One dynamic that hasn’t been researched thoroughly, yet naturally occurs in human collaboration, is for instance leader–follower dynamics. In our previous study, we tackled the question of leader–follower role allocation in human dyads during a collaborative reaching task, where the results implied that the subjects who performed higher in the individual experiment would naturally assume the role of a leader when in physical collaboration. In this study, we build upon the leader–follower role allocation study in human dyads by observing how the leader–follower dynamics change when the collaborative task becomes more complex. Here, the study was performed on a reaching task, where one subject in a dyad was faced with an additional task of obstacle avoidance when performing a 2D reaching task, while their partner was not aware of the obstacle. We have found that subjects change their roles throughout the task in order to complete it successfully, however looking at the overall task leader the higher-performing individual will always dominate over the lower-performing one, regardless of whether they are aware of the additional task of obstacle avoidance or not.

Use of Interaction Design Methodologies for Human–Robot Collaboration in Industrial Scenarios

The key concept of collaborative robotics is represented by the presence of a strict interaction between a human user and the robotic system. As such, the study of the interaction is of paramount importance for a successful implementation of the system. In this article, we propose a novel approach to address the problem of designing a collaborative robotic system for industrial applications, focusing on the characteristics of the interaction. In particular, we will propose a set of methodologies focused on interaction design, inspired by those used for the design of user interfaces. These methodologies will allow the design of collaborative robotic systems following a user-centered approach, thus putting emphasis not only on safety and adaptability of the robotic systems (which have been widely addressed in the literature), but also on the interaction experience. While the proposed methodology was developed considering general collaborative robotics applications, two real industrial case studies were considered, to instantiate the considered framework and showcase its applicability to the real-world domain. Note to Practitioners—This article aims at bridging the gap between interaction design and collaborative robotics. In particular, the proposed methodology will represent a toolset for robotic experts (researchers and system integrators), for understanding the user experience and designing the robotic system ensuring an effective interaction. In fact, while robotics experts are typically well aware of issues and methodologies related to technological and application aspects, they often tend to ignore the principles of interaction. Such principles are commonly adopted in the design of computer-based human–machine interfaces or web applications, but, to the best of the authors’ knowledge, have never been applied to the design of collaborative robotic systems for industrial applications. Hence, this article will serve as a fundamental step to bring interaction design principles into the robot integration domain.

Development and evaluation of design guidelines for cognitive ergonomics in human-robot collaborative assembly systems

Industry 4.0 is the concept used to summarize the ongoing fourth industrial revolution, which is profoundly changing the manufacturing systems and business models all over the world. Collaborative robotics is one of the most promising technologies of Industry 4.0. Human-robot interaction and human-robot collaboration will be crucial for enhancing the operator's work conditions and production performance. In this regard, this enabling technology opens new possibilities but also new challenges. There is no doubt that safety is of primary importance when humans and robots interact in industrial settings. Nevertheless, human factors and cognitive ergonomics (i.e. cognitive workload, usability, trust, acceptance, stress, frustration, perceived enjoyment) are crucial, even if they are often underestimated or ignored. Therefore, this work refers to cognitive ergonomics in the design of human-robot collaborative assembly systems. A set of design guidelines has been developed according to the analysis of the scientific literature. Their effectiveness has been evaluated through multiple experiments based on a laboratory case study where different participants interacted with a low-payload collaborative robotic system for the joint assembly of a manufacturing product. The main assumption to be tested is that it is possible to improve the operator's experience and efficiency by manipulating the system features and interaction patterns according to the proposed design guidelines. Results confirmed that participants improved their cognitive response to human-robot interaction as well as the assembly performance with the enhancement of workstation features and interaction conditions by implementing an increasing number of guidelines.

Manipulability Optimization of a Rehabilitative Collaborative Robotic System

The use of collaborative robots (or cobots) in rehabilitation therapies is aimed at assisting and shortening the patient’s recovery after neurological injuries. Cobots are inherently safe when interacting with humans and can be programmed in different working modalities based on the patient’s needs and the level of the injury. This study presents a design optimization of a robotic system for upper limb rehabilitation based on the manipulability ellipsoid method. The human–robot system is modeled as a closed kinematic chain in which the human hand grasps a handle attached to the robot’s end effector. The manipulability ellipsoids are determined for both the human and the robotic arm and compared by calculating an index that quantifies the alignment of the principal axes. The optimal position of the robot base with respect to the patient is identified by a first global optimization and by a further local refinement, seeking the best alignment of the manipulability ellipsoids in a series of points uniformly distributed within the shared workspace.

SCREW SORTING APPLICATION WITH 6 DOF UR5E ROBOT ARM

As part of the module “Introduction of Collaborative Robot Systems” at the Zittau/Görlitz University of Applied Sciences, a practical robot application using a collaborative robot and the associated human-robot interaction must be examined in a written document. The objective of the document is to design an application for sorting screws in such a way that the motion sequences of the robot are optimized. Safety aspects are to be considered. Subsequently, the concept is to be implemented theoretically and/or practically. The work carried out and the results are to be recorded in a document. In this article, the results of the proof of a student group are presented. Thereby, the article shall give an overview on the possible procedure for the conceptual design of practical robot applications.

Novel Approach Using Risk Analysis Component to Continuously Update Collaborative Robotics Applications in the Smart, Connected Factory Model

Building on the idea of Industry 4.0, new models of the highly connected factory that leverage factory-generated data to introduce cost-effective automation and involve the human worker for creating higher added value are possible. Within this context, collaborative robots are becoming more common in industry. However, promises regarding flexibility cannot be satisfied due to the challenging process of ensuring human safety. This is because current regulations and standards require updates to the risk assessment for every change to the robotic application, including the parts involved, the robotic components, and the type of interaction within the workspace. This work presents a novel risk analysis software tool that was developed to support change management for adaptive collaborative robotic systems in the connected factory model. The main innovation of this work is the tool’s ability to automatically identify where changes have been made to components or processes within a specific application through its integration with a connected factory architecture. This allows a safety expert to easily see where updates to the risk assessment are required, helping them to maintain conformity with the CE marking process despite frequent changes. To evaluate the benefits of this tool, a user study was performed with an exemplary use-case from the SHOP4CF project. The results show that this newly developed technology for risk assessment has better usability and lower omission errors when compared to existing methods. Therefore, this study underlines the need for tools that can help safety engineers cope with changes in flexible robotics applications and reduce omission errors.

An intelligent master–slave collaborative robot system for cafeteria service

In order to improve the efficiency and reduce the labor cost of cafeterias, an intelligent master–slave collaborative robot system is developed for cafeteria service in this paper. The developed system can automatically complete the tasks of scooping dishes, taking bowls and pouring dishes into the bowl based on master–slave collaboration. Specifically, a dynamic geometry feature graph convolution network (DGG) is devised using the 3D point cloud of the dish, which can efficiently predict the scooping positions of the different dishes. Moreover, a master–slave motion planning control method is proposed to achieve fast and smooth trajectories for both arms, which can accomplish the cafeteria service tasks collaboratively. Furthermore, we establish a dataset containing point clouds and color images of various Chinese food. Experiments demonstrate that the DGG network can achieve superior performance over other state-of-the-art point cloud segmentation networks. Besides, the designed robot system can well meet the requirements of operation accuracy and speed, confirming its practicality in cafeteria services.